Skip to content

improvement(agent, file-block): files in agent block, file block v4#4610

Merged
Sg312 merged 6 commits into
stagingfrom
improvement/file-block-v4
May 15, 2026
Merged

improvement(agent, file-block): files in agent block, file block v4#4610
Sg312 merged 6 commits into
stagingfrom
improvement/file-block-v4

Conversation

@Sg312
Copy link
Copy Markdown
Collaborator

@Sg312 Sg312 commented May 15, 2026

Summary

Add files to agent block, update file block

Type of Change

  • New feature

Testing

How has this been tested? What should reviewers focus on?

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link
Copy Markdown

vercel Bot commented May 15, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped May 15, 2026 5:10pm

Request Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 15, 2026

Greptile Summary

This PR adds file attachment support to the agent block (upload or wire files into any agent's latest user message) and introduces a new file_v4 block with dedicated Read, Fetch, Write, and Append operations, replacing the legacy v3 block in the toolbar.

  • A new attachments.ts module centralises per-provider content building (OpenAI Responses API, Anthropic document blocks, Gemini inlineData, Bedrock native blocks, OpenRouter, and image-only adapters); all provider files now call into it to strip the files field before sending the payload.
  • attachFilesToLastUserMessage in the agent handler hydrates user-provided files and appends them to the last user message; the early guard that should reject unsupported providers (xai, deepseek, cerebras) does not fire because getAttachmentProvider returns a non-null value for them, causing unnecessary file hydration and a test that will fail in CI.
  • file_v4 introduces a file_read tool backed by the existing /api/tools/file/manage route (new read operation) and a file_fetch tool that reuses the parse endpoint with optional custom headers for authenticated downloads.

Confidence Score: 3/5

The file attachment pipeline works correctly end-to-end for all supported providers, but the early guard in the agent handler silently passes for three declared-unsupported providers, causing wasted file hydration and a broken test.

The early rejection check in attachFilesToLastUserMessage passes for xai, deepseek, and cerebras because getAttachmentProvider returns a non-null string for them. In production the error is eventually thrown inside the provider's formatter, but only after files are fetched and decoded. In the test suite executeProviderRequest is mocked, so the formatter never runs and the new test "should reject files for providers without attachment support" will fail in CI.

apps/sim/executor/handlers/agent/agent-handler.ts and apps/sim/executor/handlers/agent/agent-handler.test.ts need the most attention.

Important Files Changed

Filename Overview
apps/sim/executor/handlers/agent/agent-handler.ts Adds file attachment support to agent blocks; early provider validation check misses xai/deepseek/cerebras providers, causing unnecessary file hydration and a broken test.
apps/sim/providers/attachments.ts New module handling per-provider content building (OpenAI, Anthropic, Gemini, Bedrock, OpenRouter, image-only providers); well-structured with correct unsupported-provider throws, but UNSUPPORTED_FILE_PROVIDERS is unexported, preventing upstream callers from doing early validation.
apps/sim/blocks/blocks/file.ts Introduces FileV4Block with Read/Fetch/Write/Append operations, hides FileV3Block from toolbar; param routing logic looks correct.
apps/sim/blocks/blocks/agent.ts Adds file-upload and advanced file subblocks to the agent block, normalized through normalizeFileInput before passing to the provider request.
apps/sim/app/api/tools/file/manage/route.ts Adds a new read operation supporting both workspace file IDs and direct file input objects, with correct 404 handling for missing IDs.
apps/sim/executor/handlers/agent/agent-handler.test.ts New test for rejecting files on deepseek/unsupported providers will fail because the guard it tests does not actually throw for those providers.
apps/sim/tools/file/parser.ts Extracts parseFileParserV3Response as a shared helper, adds fileFetchTool with headers normalization; small, clean refactor.

Sequence Diagram

sequenceDiagram
    participant UI as Agent Block UI
    participant AH as AgentBlockHandler
    participant AT as attachments.ts
    participant HY as hydrateUserFilesWithBase64
    participant PR as Provider (openai/anthropic/…)

    UI->>AH: execute(inputs.files)
    AH->>AH: normalizeFileInput(filesInput)
    AH->>AH: getAttachmentProvider(providerId) — guard
    Note over AH: xai/deepseek/cerebras pass guard (non-null) ⚠️
    AH->>HY: hydrateUserFilesWithBase64(userFiles)
    HY-->>AH: UserFile[] with base64
    AH->>AH: attach files to last user message
    AH->>PR: executeProviderRequest(messages with files)
    PR->>AT: buildXxxMessageContent / formatMessagesForProvider
    AT->>AT: prepareProviderAttachments
    Note over AT: Throws for xai/deepseek/cerebras here
    AT-->>PR: formatted content parts
    PR-->>AH: ProviderResponse
Loading

Reviews (1): Last reviewed commit: "Clean up attachments" | Re-trigger Greptile

Comment thread apps/sim/executor/handlers/agent/agent-handler.ts
Comment thread apps/sim/executor/handlers/agent/agent-handler.ts Outdated
@Sg312 Sg312 changed the title feat(agent, file-block): files in agent block, file block v4 improvement(agent, file-block): files in agent block, file block v4 May 15, 2026
@Sg312 Sg312 merged commit bad21cb into staging May 15, 2026
13 checks passed
@waleedlatif1 waleedlatif1 deleted the improvement/file-block-v4 branch May 15, 2026 18:05
waleedlatif1 added a commit that referenced this pull request May 15, 2026
…#4619)

* improvement(providers): align attachment dispatch to vendor SDK types

Post-merge audit of #4610 surfaced three follow-ups:

1. xAI Grok vision was blocked. Grok runs through the OpenAI-compatible
   chat-completions endpoint, so removing xAI from UNSUPPORTED_FILE_PROVIDERS
   and routing it through the image-only branch restores image attachments
   on vision models.

2. Azure OpenAI chat-completions deployments blocked any file attachment.
   Added a per-message image_url parts path; documents still require the
   Responses API endpoint and throw a clear, actionable error.

3. Wire shapes were loosely typed (Record<string, unknown> arrays).
   Replaced with `satisfies` clauses against each vendor SDK union at every
   push site: OpenAI Responses/Chat, Anthropic ContentBlockParam, Gemini
   Part, Bedrock ContentBlock members. AnthropicImageMediaType now derives
   from Base64ImageSource['media_type'] so it tracks SDK updates.

Also collapsed the validation cascade into an exhaustive switch with
`never` enforcement, and dropped the redundant per-provider
formatMessagesForProvider call from xai/index.ts (providers/index.ts
already runs the dispatcher centrally).

* fix(providers): restore getProviderAttachmentMaxBytes export and xAI message dispatch

- Restore `getProviderAttachmentMaxBytes` — still consumed by agent-handler.ts
  for per-provider attachment size limits in file hydration
- Restore `formatMessagesForProvider(allMessages, 'xai')` — providers/index.ts
  does NOT dispatch centrally on this branch; each OpenAI-compat provider
  formats its own messages. Without it, xAI Grok vision drops image attachments

* fix(providers): tighten ResponsesInputItem content type to SDK ResponseInputContent

Build fix: buildOpenAIMessageContent returns ResponseInputContent[] which
isn't assignable to Record<string, unknown>[] (ResponseInputText lacks an
index signature). Align the type to the SDK shape.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant